Batch processing is a way of operating electronic computers, and goes back to the time of punched cards. Batch processing is generally understood to be the processing of a series of batch jobs. Each individual batch job is usually provided with all necessary programs, data and instructions, so that it can be completely executed without any interaction by a user.
Batch jobs are defined in a control language, which is generally called JCL (Job Control Language). By means of such a control language, on the one hand execution of batch jobs can be fully planned in advance, so that they can run in the background without any interaction. On the other hand, near-system or physical data (file names, addresses of output devices, etc.) can be isolated from the running program and displaced into JCL control instructions. In this way, above all those processes which require hours or days of computing time can be flexibly planned. Such computation-intensive processes are typical for periodically occurring batch processing runs such as day end or year end processing in banks and other large companies.
Batch-processing-based operating systems often have special system components for controlling and monitoring the running of batch processing runs. These system components read the JCL control instructions and interpret them. The MVS (Multiple Virtual Storage) operating system, for instance, uses JES (Job Entry Subsystem) to read and interpret the JCL control instructions.
Today, many operating systems—in some cases through additional components—offer additional operating modes as well as batch processing, such as interactive processing or real time (online) processing. The MVS operating system for example, which is based on batch processing, becomes capable of real time processing with the addition of CICS (Customer Information Control System). The addition of TSO (Time Sharing Option) extends MVS with interactive processing. An overview of TSO, JCL and JES is given in Teuffel, Michael, TSO: Time Sharing Option im Betriebssystem MVS—das Lehr-und Handbuch für den erfolgreichen TSO-Benutzer (TSO: Time Sharing Option in the MVS operating system—the textbook and manual for the successful TSO user), 3rd edition, Munich, Oldenbourg, 1989.
As already indicated, JCL control instructions usually include near-system or physical information, and link it to the program-specific data. For instance, if a program contains a generic designation, chosen by the programmer, for a file type, in the JCL control instruction there is a reference to the file (data set) which is stored in the system, and which should be associated with the generic file designation when the program runs. In MVS/TSO, this reference is made by a JCL control instruction such as
//my_inputdata DD DSN=ed1.input.data
my_inputdata here stands for the generic designation which the programmer has chosen for a file type which the program uses, and ed1.input.data, in the example, stands for the physical name of a stored file which the program is to process. On the basis of a database catalogue, which associates the physical file name with its physical storage location (disk name, track address, etc.), during the processing of the JCL control instructions the necessary physical files can be identified, the required temporary storage areas can be reserved and the desired files can be read. The same applies if files are written.
Despite the above-mentioned advantages of batch processing, particularly in the case of computation-expensive and periodically occurring computing processes, the range of functions which have been available until now is insufficient. Thus, in many batch-processing-based operating systems, it is necessary to generate all the files which a program requires before the program starts. In practice, it has been shown that often too many empty files are generated in advance. However, unnecessary empty files are difficult to identify and are often not deleted again after the end of a batch processing run. For this reason, in traditional batch processing systems, much storage space is unnecessarily occupied by empty files which could actually be deleted. The same applies to files which were accessed in the past, but are no longer required in future program runs.
A further disadvantage of traditional batch processing mechanisms is that in the JCL control instructions the physical file names (e.g. those of the files to be written) must be fixedly allocated. In turn, a fixed physical storage space must be associated with each file name in the database catalogue. In the case of repeatedly run batch processing runs, the effect of this rigid structure is that a file to be addressed by the program is constantly overwritten, because of the fixedly allocated physical file name. Therefore, for a long time no satisfactory versioning of files was possible, i.e. different versions of a specified file type could not be created. (At least not at acceptable cost. For instance, in many cases constant allocation of new physical file names in the JCL control instructions was ruled out, because these control instructions would then have had to be changed manually before each batch processing run.)
To make simpler versioning of files possible, it was proposed that file names should be allocated dynamically at operating system level by means of Generation Data Groups (GDGs). According to this approach, in the JCL control data there is only a group name. The operating system generates a new file (data set) of a specified group only during a batch processing run.
However, the use of GDGs is unsuitable for many applications, because it is only possible to address individual files of a GDG relatively, by referring to their position in a file stack (e.g. as the “last file but one” or “seventh file” in the stack). Understandably, this makes programming difficult, and in particular it makes reading instructions difficult for a human user. Additionally, in the case of program crashes a restart is critical, because meanwhile new files may have been added to the stack or old files may have been deleted. In such cases, the relative stack position of a particular file has also changed, so that in subsequent processing the wrong file is read, deleted, overwritten, etc.
The invention is based on the object of giving an improved mechanism for dynamic management of files in a batch-processing-oriented environment. The mechanism should be suitable for simplifying file housekeeping (e.g. recognising and dealing with files which are no longer required). Also, the mechanism should allow reliable file versioning.